Field of the Invention
The invention relates generally to grid enabled computing environments, and particularly to an apparatus, method, and computer program product for dynamic security based routing, which increases the calculation and/or processing resources available for grid job processing, while maintaining specific levels of security.
Description of the Related Art
A grid job is a computer processing job that is portioned out across a plurality of processors over an unbounded network such as the Internet. Grid job processing is often employed when large processing jobs must be carried out, but only limited processing capabilities are available. For example, mathematical modeling of economic systems, global weather patterns or weapons systems involves complex calculations that require more processing resources than are typically available in conventional processors.
Large processing jobs may be carried out by dividing the job into many smaller jobs and distributing the jobs to a plurality of processors. When the processing capability of more than one processor is used to carry out complex processing tasks, the processing resources are known as shared resources. Utilizing shared resources to carry out processing of data that may undergo analysis in a non-sequential manner is an especially attractive use of shared processing.
Large processing jobs may also be carried out on a single processor capable of carrying out complex processing tasks. Individual, self-contained computers capable of carrying out complex processing are often referred to as “supercomputers” and include those made by the Cray Corporation. Supercomputers are relatively few in number.
The processing capability of a supercomputer is defined by the number of calculations that the supercomputer can carry out per unit of time. The total computing time available on a supercomputer is limited by the demand for the computer's processing resources and the number of calculations per unit of time that the computer can carry out. The demand for computing time is usually greater than the processing capability of the computer. If supercomputers were the only resource available for carrying out complex processing, it would not be possible to meet the demand for complex processing and a severe shortage of complex processing resources would be evident.
Shared resources can be used to carry out complex processing by utilizing the processing capabilities of many smaller computers in parallel. Cumulatively, the processing capabilities of many smaller processors may be equivalent to the processing capabilities of a single supercomputer. Purchasing many smaller computers instead of a single supercomputer may not be an economically acceptable way to carry out complex processing because the costs associated with the purchase of many individual computers is greater than the cost of a single supercomputer.
The processing resources of most individual computers are only partially used. Computer technologists have found that this unutilized or underutilized processing capability can be used at little or no cost. These processing resources are accessible, for example, over networks such as the Internet or intranets that inexpensively interconnect many computers. By bundling the processing capabilities of many individual computers, e.g., by connecting the individual computers in a network, it is possible to assemble the processing capability needed to carry out complex calculations and avoid the need for purchasing an expensive supercomputer-type processor.
The sharing of processing capability over many computers connected to one another over an unbounded network is referred to as grid computing. The term “grid” is used to represent a physically interconnected network of individual processors, each having a certain processing capability.
Grid computing is thought to have started within universities. Universities often lacked the internal computing resources needed to carry out the calculations needed to model complex systems. To overcome this lack of resources, universities began to share their computing resources with each other. For example, a first university may have 50 servers and a second university may have 50 servers. At any given point in time, a processing job could be portioned among the 100 servers of the first and second universities. By using such grid systems both the first and second universities could better schedule and apportion their available processing time and gain greater efficiency from their computer processing resources. The degree to which additional exterior resources (e.g., the processing capability separate from the processing capability of a central computer or core of computers) can be interconnected to carry out processing is referred to as parallelism.
Grid computing suffers from a significant shortcoming in that the security and/or privacy of the data that is processed with the use of external resources (e.g., shared resources and/or processing capability) cannot be guaranteed. Data sent outside of a secure network may be subject to a substantial risk of interception, disclosure and/or corruption.
With heightened emphasis being placed upon maintaining the security (e.g., the confidentiality) of certain types of information such as personal information, it has become more difficult to utilize shared processing resources for certain applications. Such security-sensitive applications include the transfer, storage and/or use of financial data such as the individual financial transactions and/or obligations associated with a particular individual.
As a greater amount of secure data is generated more methods for analyzing this data are developed. This in turn provides richer databases requiring even greater security protection. As the amount of security-sensitive information and/or data increases, there is a greater need for improved security.
It is desirable to use shared resources to the greatest extent possible in order to achieve greater processing efficiency; however, there is a concurrent need to carry out the processing of sensitive data only within environments wherein the data undergoing processing is safe from disclosure, corruption, and/or interception by any party other than the party submitting a processing job.
True grid computing (e.g., unbounded utilization of shared resources over the Internet) is not widely used today due in part to impediments presented by the necessity for interconnecting disparate processing resources. In a conventional grid, different processing resources are interconnected by conventional network routing which is “specific permission” routing. The security and organizational protocols used with specific permission routing generally allow network traffic (e.g., communications between different processors) to flow only when the traffic is permitted by a routing apparatus (typically a router or a firewall). A firewall is a gateway that limits access between networks in accordance with a local security policy. The two networks separated by the firewall are in two separate computer environments.
Communications between processing resources usually occurs by sending “packets” of digitized information from one processor to another processor over a network connection. The packets contain a portion of the total information (e.g., the data undergoing processing) of the processing job. Each packet is identified by a destination address and an origin address, in addition to other information. The packet must include sufficient information to be recognized by the router as it leaves and/or enters a network.
A network may include as few as two computers or may include an unlimited number of computers, such as the environment of computers interconnected by the Internet. The internetworking of computers may be accomplished through a series of hubs (e.g., routers and/or servers). The communication taking place over a network is typically controlled by a router which organizes the information flow so that communication between many computers can be directed seamlessly over a single connection (e.g., telephone and/or data line).
Routing is an important determinant of the characteristics of any process or method that utilizes shared processing resources. A restrictive routing scheme is preferred for processing of information that requires a high level of security. High security can be achieved through the routing infrastructure. For example, a firewall may be used to restrict the egress of the data outside of a defined computing environment such as a grid cluster, and likewise prevent the ingress of data which might infect or corrupt the data undergoing processing.
In conventional routing, the router or firewall is programmed to recognize specific addresses or a specific range of addresses and based on this identifying information, permits packets having such addresses to flow across the firewall. Packet information typically has a destination address, which allows routing the packet information routed through various routers to its final destination. If a firewall allows an information packet to pass (i.e., the address of the packet falls within the specific range of permissible addresses or is a specifically permitted address), the firewall allows the packet to communicate with another processor located outside of the network.
A router may be configured so that communication between processing resources (e.g., http based traffic) is permitted only within the confines of a network that includes only a limited number of processors each uniquely identifiable (e.g., a bounded grid known as a grid cluster). Communication with processors outside of the network is not permitted by the router which is programmed to permit communication only between certain predefined destinations (e.g., only between the uniquely identifiable computers of the network). Data packets containing address information identifying a destination (e.g., processor) outside the predefined network are not permitted to exit the defined network.
Conventional routing is static meaning that once a routing rule is in place, all network traffic must conform to that rule. Static routing is the most widely used method of network traffic routing because it allows for good security control over networks and any attached devices.
Static routing security is not optimal for grid-enabled processing. In a true grid environment, the processing workload is spread as broadly as possible across as many networks and devices as possible in order to maximize job resources, parallelism, and performance. It is difficult to reconcile the need to increase parallelism with traditional routing methods because the inclusion of additional processing resources in a network may compromise the security of the network.
Although the advantages of grid computing are now being more widely recognized, limitations such as grid specific routing techniques and security concerns, have restricted most efforts at grid processing to clusters of processing resources instead of unlimited grid environments such as the Internet. Any computing cluster has grid resources, such as computers, that are typically secured within a corporate subnet, intranet or network. The routing rules of such a computer cluster usually do not allow grid traffic to flow freely among all possible grid resources, as would be the case in a true grid environment (e.g., unsecured Internet communication).
FIG. 1 contrasts a grid-cluster with a true grid environment (e.g., an unrestricted grid environment). Grid-cluster 10 consists of only enterprise subnet or network 1 and enterprise subnet or network 2. Enterprise subnet or network 1 includes corporate computers 1A, 1B, 1C, and corporate router 3A. Enterprise subnet or network 2 includes computers 2A, 2B, 2C, and corporate router 3B. These subnets or networks, and associated resources, may be physically separated from other processors, e.g., located within the confines of the corporation, or may be separated from other processors by certain security protocols. The defining characteristic of grid-clusters is that they keep all traffic within the walls of the corporation. Internet routed networks and resources 12 provide those components that must be added to a grid-cluster 10 in order to achieve true grid computing. True grid computing is realized when a grid job is spread to all available resources, including resources (e.g., processors) physically separated from the enterprise or separated from the enterprise by a security protocol or screen. As shown in FIG. 1, with the addition of Internet routed networks and resources 12, a grid job originating from the corporation having a subnet, intranet, or network 10, is spread via router 5 to resources available over the Internet including computers 4A, 4B, and 4C.
As shown by FIG. 1, to employ true grid computing, a processing job must be communicated beyond the resources of its originator (e.g., a company or a company's internal grid-cluster) to the Internet. This cannot be achieved by conventional network routing which functions to halt specific traffic from passing through a firewall or router.
As mentioned above, conventional static routing is unsuitable for grid enabled processing. Grid enabled processing preferably includes the capability to dynamically locate and identify processing resources outside of an internal grid-cluster. Such dynamically located and identified processing resources may not have a known address when a grid job is started. A packet that is a part of a grid job does not necessarily include an address but may still communicate with other processing resources. In the case of conventional routing, the absence of an address causes the router to block communication by prohibiting packets from leaving the electronic or physical confines of an internal grid-cluster.
Grid-cluster computing is commonly mistaken for true grid computing. True grid computing includes the free flow and free use of any resource on the Internet. Grid clusters or multiclusters are not true grid computing because they do not utilize (nor have the capability to utilize) or communicate with resources that are not part of a defined cluster.
The increasing demand for computer processing resources has created a need for ways to better manage and maximize existing processing resources. Substantial economies of scale may be realized by better utilizing or reducing the amount of computer processing resources needed to fulfill the processing requirements of many business, governmental and academic users. Rather than buying new, expensive, specialized equipment to increase processing power, grid computing may be used to distribute processing jobs over a plurality of processors and thus allow fewer or less expensive processing resources to be purchased and maintained. A system that securely uses a plurality of computers for a particular processing job may also increase the speed for completing any particular processing job. A job that may take three weeks using only the limited resources of a single processor or a grid cluster may be completed in as little as 24 hours by using methods and equipment to better manage existing processing resources.